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ABSTRACT 

This paper presents findings of a study on the gap between 
high-school academic content standards and the knowledge and skills needed 
for success in entry-level university courses. The study had two parts. The 
first described the development of standards of knowledge and skills for 
university success (KSUS) through a combination of a modified version of the 
Delphi method and content analysis to establish their validity as a reference 
point for analyzing state academic-content standards. The KSUS standards were 
developed from data collected from 400 faculty members and administrators 
from 20 universities. Additional data were derived from content analysis of 
examples of student work. The second part of the study employed content- 
alignment analysis in a pilot study of one state *s standards in reading, 
mathematics, and science: The state of Washington's essential academic 
learning requirements were analyzed against the KSUS standards. The results 
are described in narration, supplemented in tabular form. The KSUS, included 
in an appendix, represent the only comprehensive statement of university 
entry-level skills in a standards-like format. The pilot analysis 
demonstrated one strategy that can be employed to identify the degree of 
alignment between state and university academic-content standards. (Contains 
15 references.) (WFA) 
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This paper presents research findings generated by Standards for Success (S4S), a 
collaborative research project sponsored by the Association of American Universities 
(AAU) and by The Pew Charitable Trusts. The goal of the research is to identify the gap 
between high school academic content standards developed over the past decade and the 
knowledge and skills needed for success in entry-level university courses. Are 
expectations aligned across the two systems? Will state academic content standards help 
prepare students to succeed in entry-level university courses? What effects on the 
university general education curriculum will standards for entry-level courses have? 
Theoretical Framework 

This study is situated in relation to certain established concepts from 
organizational theory in the area of organizational linkages. Weick, in particular, has 
established a number of important constructs related to linkages across organizational 
boundaries, most notably the concept of “loose coupling” (Weick, 1976). Standards- 
based approaches call into question Weick’ s contention that educational institutions in 
the United States are by their very nature loosely coupled. 

Standards have a potential norming effect on educational programs across 
institutional boundaries. DiMaggio and Powell (1983) highlighted the power and 
influence of “institutional isomorphism,” the tendency of organizations to look to other 
similar organizations for examples of normative behavior and limits of legitimacy. Levitt 
and March (1988) continued and expanded this theoretical concept to broader notions of 
organizational learning and the effect of common meaning on such learning. The 
standard-setting activities undertaken in this study are designed to create a point of 
legitimacy that would allow for connections between high school standards and 
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university standards. If this succeeds, loose coupling theories will need to be reexamined 
to take into account formal linkages of this nature. Standards-based reform will have 
demonstrated the power of institutional isomorphism if schools accept standards across 
system boundaries as a common reference point. 

A second, related body of theory on organizational alignment has arisen from the 
policy literature. Most notable is the concept of policy coherence (Fuhrman, 1993), which 
states that educational systems that receive consistent policy messages focused on student 
learning will perform better than those that do not. The signals that are sent by formal 
policies to organizations and across organizational boundaries within a linked system 
such as education have potentially significant effects on organizational functioning. 
Emerging theories that seek to explain the student transition from high school to college 
(Bragg & National Center for Research in Vocational Education Berkeley CA., 1999) and 
how state policy affects connections between high schools and colleges (Conley, in press) 
also serve to ground the study’s conceptual framework. 

Methods of inquiry and data sources 

This is a multi-method study in two parts. Part one describes the development of 
Knowledge and Skills for University Success (KSUS) standards through a combination 
of a modified version of the Delphi method and content analysis in order to establish their 
validity as a reference point for analyzing state academic content standards. The second 
part of the study employs content alignment analysis in a pilot study of one state’s 
standards in Reading, Mathematics, and Science. 

The KSUS standards were developed from data collected initially through nine 
meetings at leading American research universities that are members of the AAU. 
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Participants came from a total of 20 AAU universities. The meetings were known 
collectively as the National Conversation on Key Knowledge and Skills for University 
Success. Sessions were designed to generate two types of statements: 1) key knowledge 
by discipline; 2) cognitive skills and abilities that cut across disciplines. Trained 
facilitators (usually a faculty from hosting institution and the particular discipline) began 
each session by asking the following two questions within each group: 1) What content 
knowledge do students need to have to be successful in your entry-level course? 2) What 
are the more general cognitive skills that students need if they are to be successful in a 
university environment? 

Data collected at the National Conversation meetings were subjected to a 
modified version of the Delphi method, a technique for reaching consensus among 
experts on a topic (Tinstone & Turoff, 1975). Nine National Conversation meetings were 
held at universities that are members of the Association of American Universities (AAU). 
Approximately 400 faculty members and administrators who work with incoming 
students identified the key knowledge and skills needed for success in entry-level courses 
at the 20 institutions represented at the meetings. Participating universities included the 
Universities of California, Berkeley; Illinois; Iowa; Michigan; Minnesota; Nebraska; 
Wisconsin; Harvard University; Indiana University; New York University; Pennsylvania 
State University; Rutgers, and the Massachusetts Institute of Technology. 

The input received at each successive meeting was analyzed and compiled, then 
sent to participants in that meeting for their review. Results from the first six meetings 
were then subjected to topical analysis to identify elements that appeared most 
frequently. These preliminary findings were reviewed by Mid-Continent Research for 
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Education and Learning (McREL), an external consulting group with expertise in 
academic content standards, with the goal of identifying areas of congruence with 
existing state and national content standard documents and of stating the preliminary 
results in language that paralleled such standards. These revised knowledge and skill 
statements were then reviewed at three subsequent meetings at AAU universities. The 
scope, magnitude and frequency of recommended changes decreased at each subsequent 
meeting until many of the suggestions were for changes in grammar, formatting, or 
individual word choice. This revised set of standards was then submitted to a Content 
Review Panel consisting of professors and instructors from AAU universities with 
particular expertise and content knowledge, most of whom had previously participated in 
a National Conversation meeting at their campus. The version of the standards that 
incorporated the Content Review Panel’s recommendations was labeled the draft KSUS 
standards. These were then sent to all 400 participants in the National Conversation as 
well as to AAU university presidents, who were asked to review the standards. Minor 
changes were made based on suggestions received from these reviewers. The draft KSUS 
standards where the versions used for analyzing state assessments. The final version of 
the Knowledge and Skills for University Success incorporated minor changes suggested 
by raters. The final version was ultimately endorsed by a total of 28 AAU institutions out 
of its 60 American members. 

Additional data were derived from content analysis of examples of student work 
submitted by faculty from participating institutions. Using a rating sheet, participants 
scored each piece of work individually to rate how representative it was of desired key 
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knowledge and skills. Work samples were then linked to specific KSUS standards to 
illustrate the cognitive challenge associated with the KSUS. 

Course outlines and syllabi were a final source of data to triangulate and validate 
the KSUS standards. University faculty from participating institutions submitted these 
artifacts from their entry-level courses. Content analysis was employed to ascertain the 
specific knowledge and skill required to succeed in the course and the match between 
course requirements and the KSUS standards. This process helped validate that the 
knowledge and skills faculty said were needed for success in university courses were 
actually evident in course syllabi and assignments. Participants also analyzed and rated 
state high school content standards from one to three states, including their home state. 
Using a three-point scale, participants rated each standard statement as “critical,” 
“useful,” or “not useful” for success in entry-level university courses. 

As a summary activity, project staff and national consultants analyzed the “fit” 
between the KSUS standards and individual state standards using a process adapted from 
Webb (1997, 1999, 2002) in which cognitive complexity, range and breadth of coverage 
were ascertained. 

Results 

Key knowledge and skills standards were expressed in three formats: 1) a 
taxonomy of key academic content knowledge and skills in five disciplines; 2) a narrative 
statement of the cognitive skills that cut across disciplines including the habits of mind 
that students should seek to develop; and 3) a set of examples of student work from 
university courses designed to illustrate the challenge level associated with each KSUS 
statement. 
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The KSUS standards have three levels of detail: Standard, objective, sub- 
objective. Length restrictions prevent an exposition of all the KSUS standards for all 
disciplines. An example from mathematics and English is presented below: 

II. ALGEBRA 

A. The student will know and apply basic algebraic concepts. 

A. 1 . Use the distributive property to multiply polynomials. 

A.2. Divide low degree polynomials (e.g., long division). 

A.3. Factor polynomials (e.g., difference of squares, perfect square trinomials, 
difference of two cubes, and trinomials like x^ + 3x + 2). 

A.4. Add, subtract, multiply, divide, and simplify rational expressions 
including finding common denominators. 

A.5. Understand properties and basic theorems of exponents and roots (e.g., 
(x^)(x^)=x^ and (Vx)^ = x^^^). 

A.6. Understand properties and basic theorems of logarithms (to bases 2, 10, 
and e). ' 

A.7 Know how to compose and decompose functions and find inverses of 
basic functions. 

I. READING AND COMPREHENSION 

A. The student will use reading skills and strategies to understand literature and 
informational texts. 

A. 1 . Use reading skills and strategies to understand a variety of informational 
texts: instructions for software, job descriptions, college applications, 
historical documents, government publications, newspapers, textbooks. 

A.2. Use monitoring and self-correction methods and know when to read aloud. 

A.3. Engage critically with the text: annotating, questioning, agreeing or 
disagreeing, summarizing, critiquing, formulating own responses. 

A.4. Understand narrative terminology: author versus narrator, historical versus 
implied author, historical versus present-day reader. 

A.5. Use reading skills and strategies to understand a variety of types of 
literature: epic piece flliadi or lyric poem, narrative novels, and 
philosophical pieces 

A.6. Understand plot and character development in literature, including 
characters’ motives, causes for actions, and the credibility of events. 

A.7. Understand vocabulary and content: subject-area terminology, connotative 
and denotative meanings, idiomatic meanings. 

A.8. Understand basic beliefs, perspectives, and philosophical assumptions 
underlying an author’s work: point of view, attitude, or values conveyed 
by specific use of language. 

A.9. Use a variety of strategies to understand the origins and meanings of new 
words: analyzing word roots and affixes, recognizing cognates, using 
context clues, determining word derivations. 
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A. 10. Make supported inferences and draw conclusions based on textual 
features: evidence in text, format, language use, expository structures, 
arguments used. 

Major headings from the cross-cutting skills and attitudes include but are not 
limited to the following: writing skills, analytic and critical thinking skills, 
inquisitiveness, ability to accept criticism, willingness to edit and revise work to reach 
high quality standards, research skills, ability to evaluate information critically and form 
opinions based on information, intellectual curiosity, openness, ability to work with 
peers, ability to use technology successfully. 

English, mathematics and second languages standards captured a relatively clear 
and distinct set of attributes associated with each respective discipline. Science and social 
sciences reflected the relative complexity of these areas, each of which comprises a series 
of distinct academic disciplines. The standards in these two content areas were grouped 
into the skills that cut across the disciplines within the area along with accompanying 
listings of the key knowledge attributes for each discipline within the area. 

Pilot Analysis of One State’s Standards Usin^ KSUS 

The KSUS standards were used in a pilot study to ascertain the relative challenge 
level of one state’s standards in comparison with university expectations. The state of 
Washington’s Essential Academic Learning Requirements (EALRs) in Mathematics, 
Reading, and Science were analyzed against the KSUS standards in Mathematics, 
English, and Natural Sciences. 

The data were subjected to a content alignment analysis methodology (Webb, 
1999; Impara, 2001). Each state standard was compared with the KSUS standards to 
determine the matches that existed. This is known as Categorical Concurrence. Based on 
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the set of standards for which concurrence was found, a second analysis was conducted. 
The Depth of Knowledge for each state standard was determined and then compared to 
the depth of knowledge for the corresponding KSUS standard. Through this process it 
was possible to ascertain the degree to which first, the standards aligned, and second, the 
degree to which cognitive challenge was properly sequenced between the state standards 
and the KSUS standards. Range and breadth of knowledge were also computed, but are 
not included in this paper. 

The depth of knowledge categories are based on those developed by Marzano 
(2001). Marzano’s New Taxonomy is hierarchical — one level builds off of another, so 
that each level requires progressively more cognitive skill, effort, and sophistication. The 
hierarchy is based on the conception that each level requires more sophisticated 
processing in short-term memory before information is moved to long-term memory. 
This empirically derived framework is based on brain research and on cognitive and 
information processing sciences. See Appendix A for a detailed description of the model. 

The New Taxonomy consists of the following six levels: 

1. Retrieval 

2. Comprehension 

3. Analysis 

4. Utilization 

5. Goal setting and monitoring 

6. Self system thinking 
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Only the first five levels were used to rate the EALRs. The sixth, self system 
thinking, does not have a comparable category in the KSUS against which the EALRs 
were compared. 

Five raters with experience teaching entry-level university courses with 
backgrounds and expertise in standards system development and rating were trained 
using a technique similar to that used to train Advanced Placement scorers. Through such 
training, discussion, and guidance by the facilitator with reference to scoring protocols 
and criteria, the raters were able to attain an inter-rater reliability on the math EALRs of 
.68 at 4th grade, .69 at 7th grade, and .81 at 10th grade, demonstrating adequate 
consistency in their judgments at the 4th- and 7th-grade levels and strong consistency at 
the lOth-grade level. These reliabilities are representative of the range in ratings 
evidenced in similar rating activities. 

Mathematics 

The comparison of the Mathematics K-10 EALRs to the university-level 
Knowledge and Skills for University Success (KSUS) in Mathematics described in 
Appendix B reveals a consistent correspondence between Benchmark Level 3 lOth-grade 
Mathematics EALRs and the university-level Knowledge and Skills for University 
Success (KSUS) in Mathematics. Twenty-seven of the K-10 EALRs for which a 
correspondence was found with a university-level KSUS matched at the same rating 
level. In 25 cases, the K-10 EALR was higher than the corresponding university-level 
KSUS, and in 8 cases, the K-10 EALR was lower than the corresponding university- 
level KSUS. Thirty EALRs in math did not have a corresponding KSUS aligned with 
them. 
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The categorical concurrence index for Mathematics is .67, which exceeds the .50 
criterion level necessary to state that the concurrence between the two sets of standards is 
adequate. The depth of knowledge ratio was 25:27:8: progressive cognitive challenge, 
equal cognitive challenge, inverted cognitive challenge. By this measure, the challenge 
progression was appropriate for 85% of the EALRs in relationship to the KSUS. These 
two findings indicate the Washington EALR content standards in Mathematics meet the 
criteria necessary to designate them adequately aligned with the KSUS Mathematics 
standards. 

Table 1 contains an analysis of percentage of content items (benchmarks/ 
objectives) across topics into which the Mathematics EALRs fall in comparison to the 
weighting that other national reports and the KSUS Mathematics standards give to these 
topics. 

Table 1. Percentage of Mathematics Content Items (Benchmarks/Objectives) Across 
Topics: Grade 10 



Grade 10 


EALRs 


NCTM 


NAEP 


New 

Standards 


KSUS 


Number Sense 


13 


21 


10 


15 


17 


Measurement 


12 


6 


28 


8.5 


2 


Geometry 


9 


14 


15 


11 


16 


Trigonometry 


0 


1 


2 


1 


5 


Probability & 
Statistics 


13 


22 


25 


19 


4 


Algebra 


9 


18 


35 


19 


27 


Problem Solving 


15 


5 


0 


5 


14 


Reasoning 


10 


5 


0 


5 


Math Language 


9 


4 


0 


13 


8 


Connections 


10 


4 


0 


7.5 


2 


Total 


100 


100 


100 


100 


100 
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Reading 



The comparison of the lOth-grade Reading EALRs to the university-level 
Knowledge and Skills for University Success described in Appendix C reveals a 
consistent relationship between Benchmark Level 3 lOth-grade Reading EALRs and 
university-level Knowledge and Skills for University Success (KSUS) standards in 
English. Thirty-two of the EALRs for which a correspondence was found with a KSUS 
match at the same rating level. In eight cases, the lOth-grade Reading EALRs are higher 
than the corresponding university-level KSUS in English; in five cases, the lOth-grade 
EALRs are lower than the corresponding university-level KSUS. Fourteen lOth-grade 
EALRs in Reading do not have a corresponding university-level KSUS in English that 
aligns with them. 

The categorical concurrence index for Reading is .76, which exceeds the .50 
criterion level necessary to find that the concurrence between the two sets of standards is 
adequate. The depth of knowledge ratio was 8:32:5: progressive cognitive challenge, 
equal cognitive challenge, inverted cognitive challenge. By this measure, the challenge 
progression was appropriate for 90% of the EALRs in relationship to the KSUS. These 
two findings indicate the Washington EALR content standards in Reading meet the 
criteria necessary to designate them adequately aligned with the KSUS English standards. 

Table 3 contains an analysis of percentage of content items (benchmarks/ 
objectives) across topics into which the Reading EALRs fall in comparison to the 
weighting that other national reports and the KSUS English standards give to these 
topics. 
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Table 3. Percentage of Reading Content Items (Benchmarks/ Objectives) Across 
Topics: Grade 10 



Grade 10 


EALRs 


CBE 


NAEP 


New Standards 
High 


KSUS 


Comprehension 


40 


11 


* 

* 

o 


17 


19 


Informational Texts* 


26 


22 


65 


45 


25 


Literature 


14 


50 


35 


38 


50 


Evaluating Reading 
Progress 


10 


0 


0 


0 


0 


Word Meaning 


10 


17 


Q** 


0 


6 


Total 


100 


100 


100 


100 


100 



Science 

The comparison of the Science lOth-grade EALRs to the university-level 
Knowledge and Skills for University Success in Natural Science described in Appendix 
D reveals a consistent relationship between Benchmark Level 3 lOth-grade EALRs and 
the university-level Knowledge and Skills for University Success (KSUS). Thirty-one of 
the EALRS for which a correspondence was found with one or more KSUS matched at 
the same rating level. In 4 cases, the grade 10 EALR was higher than the corresponding 
university-level KSUS and in 13 cases, the lOth-grade EALR was lower than the 
corresponding university-level KSUS. Three science EALRs did not have a 
corresponding KSUS that aligned with them. 

The categorical concurrence index for Science is .96, which greatly exceeds the 
.50 criterion level necessary to state that the concurrence between the two sets of 
standards is adequate. The depth of knowledge ratio was 4:31:13: progressive cognitive 
challenge, equal cognitive challenge, inverted cognitive challenge. By this measure, the 
challenge progression was appropriate for 73% of the EALRs in relationship to the 
KSUS. These two findings indicate the Washington EALR content standards in Science 
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meet the criteria necessary to designate them adequately aligned with the KSUS Natural 
Science standards. 

Table 3 contains an analysis of percentage of content items (benchmarks/ 
objectives) across topics into which the Science EALRs fall in comparison to the 
weighting that other national reports and the KSUS Natural Science standards give to 
these topics. 

Table 3. Percentage of Science Content Items (Benchmarks/Objectives) Across 
Topics: Grade 10 



Grade 10 


EALRs 


NSES* 


NAEP* 


AAAS* 


KSUS 


Physical Science 


21 


29 


13 


18 


44 


Earth/Space Science 


17 


14 


13 


7 


7 


Life Science 


19 


30 


13 


21 


31 


Scientific Inquiry 


10 


6 


46** 


5 


1 


Nature of Science 


12 


7 


9 


2 


6 


Historical Perspectives 


2 


4 


27 


1 


Science, Society, and 
Technology 


7 


8 


9 


10 


Technology 


12 


2 


6 


11 


1 


Total 


100 


100 


100 


100 


101 



*9-12 grade range 
**includes practical reasoning 



Conclusion! significance 

This study is significant for the following reasons: 1) the KSUS represent the first 
and only comprehensive statement of university entrance-level skills that is presented in a 
standards-like format, rather than in terms of required courses or broad generalizations 
about general academic preparation. Interestingly, and somewhat ironically, higher 
education faculty have not been involved systematically in standards development 
(except as experts on what K-12 should do) as states have rushed to put content standards 
and assessments in place. 
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The KSUS standards and accompanying analyses of state standards can be useful 
to states as they seek to develop better standards and to align their assessments more 
closely with university entrance expectations. Given that few state standards development 
processes paid much attention to university requirements, it seems likely that as these 
systems begin to influence high school instruction, states will wish to remedy this 
potential deficit, if for no other reason than to cause students to take state assessments 
more seriously. State assessments are beginning to take hold and drive teaching. If state 
standards and assessments are not well aligned with university admissions expectations, 
the likelihood is increased that high schools will increasingly become divided into two 
communities; students who have passed state assessments and can focus on a college 
preparatory curriculum, and students who have not passed the assessments and who are 
focused on a test-preparation curriculum. Given that 63 percent of students go directly 
from high school to some form of postsecondary education (Conditions of Education 
2001), the pressures within the high school curriculum will grow for some sort of 
resolution of two potentially competing systems. The need will be greatest at inner-city 
schools where the gap between performance on state tests and high school grades is the 
most dramatic (Wolf, 1998). In such schools, students run the risk of meeting university 
entrance requirements based on the courses they take and grades they receive, but still 
being unable to pass state assessments. Such students are poorly served by both state 
standards and university entrance requirements in such a situation. 

The pilot analysis of state standards demonstrates one strategy that can be 
employed to begin to identify the degree of alignment between state and university 
academic content standards. Systematic content analysis can identify the degree to which 
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alignment exists and the specific areas in which expectations may differ. The results can 
then be fed back into the standards development process as states revise their standards 
over time. This is important to do because state standards form the domain from which 
state assessment items must be selected. If state assessments are ever to align in any 
significant fashion with the requirements for postsecondary success, state content 
standards must first be aligned. 

The third area of significance pertains to universities themselves. These findings 
will help universities to gauge the discrepancy or match between what they say they want 
their incoming students to be able to do and what actually occurs in the freshman 
curriculum. These broadly generalizable findings extend beyond any individual 
institution and can serve as a resource for individual campuses as they review their 
general education programs of study. This is important to do because university standards 
should not be expressions of “wish lists,” but rather empirically grounded descriptions of 
what is actually required to succeed subsequent to admission. Specific standards create 
accountability for universities to ensure that entry-level courses actually build upon the 
knowledge and skills identified as key prerequisites to success. 

The fourth and final area of significance of the study is its contribution to the 
extremely limited research base that exists on the topic of the connections between high 
school and college expectations. Most studies in this area seek to examine the effects of 
course requirements or completion of various courses of study on college performance 
and utilize the metric of predictive validity. Few have explored the perceptions of faculty 
or analyze the actual practices taking place in college classrooms. Even fewer have 
analyzed the relationship between state high school standards and university success 
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standards. This area of analysis is now opened up much wider by the existence of the 
Knowledge and Skills for University Success standards and emerging analytic techniques 
of the type demonstrated in this study and in similar studies in which state assessment 
items were analyzed in relation to KSUS standards (Conley & Brown, 2003). 

If American classrooms are being influenced by state standards and assessments, 
it is increasingly important to understand the expectations and standards of universities in 
particular so that the energy students and teachers expend meeting standards is consistent 
with the future plans of students who plan to attend postsecondary learning. This 
understanding of differential expectations is also important to policy makers who oversee 
standards systems and their evolution. 
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Appendix A; Summary; Depth of Knowledge Categories 

Level 1; Retrieval 
Recall: 

The standard requires the student to identify or recognize features of information, but does not necessarily 
require understanding of the structure of knowledge or of the ability to differentiate critical from non-critical 
components. 

Execution: 

The standard requires the student to perform a procedure without significant error, but does not necessarily 
require that the student understand how and why the procedure works. 

Level 2; Comprehension 
Synthesis: 

The standard requires the student to identify the basic structure of knowledge and the critical, as opposed to non- 
critical characteristics of that structure. 

R epresentation : 

The standard requires the student to identify or recognize features of information, but does not necessarily 
require the student to understand the structure of knowledge or require that the student be able to differentiate 
critical from non-critical components. 

Level 3; Analysis 
Matching: 

The standard requires the student to identify important similarities and differences between knowledge. 
Classifying: 

The standard requires the student to identify superordinate and subordinate categories related to knowledge. 
Analyzing Error: 

The standard requires the student to identify errors in the presentation or use of knowledge. 

Generalizing: 

The standard requires the student to construct new generalizations or principles based on knowledge. 

Specifying: 

The standard requires the student to identify specific applications or logical consequences of knowledge. 

Level 4; Utilization 
Decision Making: 

The standard requires the student to use the knowledge to make decisions or expects the student to be able to 
make decisions about the use of the knowledge. 

Problem Solving: 

The standard expects the student to use the knowledge to solve problems or to solve problems about the 
knowledge. 

Experimental Inquiry: 

The standard requires the student to use the knowledge to generate and test hypotheses or to generate and test 
hypotheses about the knowledge. 

Investigation: 

The standard requires the student to use the knowledge to conduct investigations or to conduct investigations 
about the knowledge. 

Level 5; Goal Setting and Monitoring 
Goal Setting: 

The standard requires the student to set a plan for goals relative to the knowledge. 

Monitoring Process: 

The standard requires the student to monitor the execution of the knowledge. 

Monitoring Clarity: 

The standard requires the student to determine the extent to which he or she has clarity about the knowledge. 
Monitoring Accuracy : 

The standard requires the student to determine the extent to which he or she has an accurate understanding of the 
knowledge. 

Note: Adapted from Designing a New Taxonomy of Educational Objectives (2000), Experts in Assessment 
Series, by Robert Marzano, Thousand Oaks, CA: Corwin Press. 
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Appendix B: Concurrence and Depth of Knowledge Between 10*** Grade 
Mathematics EALRs and KSUS Mathematics Standards 





DOK 




DOK 


K-10 Mathematics EALRs 


EALR 

Rating 


University-level KSUS 


KSUS 

Rating 




(1-5) 




(1-5) 


understand and use properties and 
symbolic representations of rational 


2 


IA.3. use radicals correctly. 


2 


numbers, powers, and roots 




IA.6. know terminology for complex numbers, 
integers, rational numbers, irrational numbers, 
and complex numbers. 


1 


understand concepts of and use 


2 


lA.l. apply arithmetic operations with fractions 


1 


processes involving prime and 




and integers (e.g., add and subtract by finding a 




composite numbers, factors and 




common denominator, multiply and divide. 




multiples, and divisibility 




reduce, and perform long division without a 
calculator). 




understand operations on rational 
numbers, powers, and roots 


2 


IA.2. use exponents and scientific notation. 


1 






IA.3. use radicals correctly. 


2 


compute with rational numbers, 
powers, and roots 


1 


IA.2. use exponents and scientific notation. 


1 






IA.3. use radicals correctly. 


2 


use estimation to predict computation 
results and to determine the 


3 


IA.4. understand relative magnitude. 


1 


reasonableness of answers involving 
real numbers, /or example, estimating 




VD.3. recognize the accuracy of an estimation. 


3 


understand how changes in 


2 


IVA.2. know how to figure area and perimeter 


1 


dimension affect perimeter, area, and 
volume 




of basic figures. 




measure objects and events directly 


2 


IVA.7. know basic formulas for volume and 


1 


or use indirect methods such as 
finding the volume of a cone given its 
height and diameter 




surface area for three-dimensional objects. 




calculate rate and other derived and 


2 


lA.l. apply arithmetic operations with fractions 


1 


indirect measurements 




and integers (e.g., add and subtract by finding a 
common denominator, multiply and divide, 
reduce, and perform long division without a 
calculator). 








lE.l. recognize which type of expression best 
fits the context of a basic application (e.g., linear 
equation to solve distance/time problems; 
quadratic equation to explain the motion of a 
falling object; or compound interest as an 
exponential function). 


3 


understand and use properties of 


3 


IVA.l. know properties of similarity. 


2 


symmetry, congruence, and similarity 




congruence, and parallel lines cut by a 
transversal. 

IVA.5. use similar triangles to find unknown 
angle measurements and lengths of sides. 


3 
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K-10 Mathematics EALRs 


DOK 

EALR 

Rating 

(1-5) 


University-level KSUS 


DOK 

KSUS 

Rating 

(1-5) 


understand and use coordinate grids 


2 


I VB . 1 . know geometric properties of lines 

(e.g., slope and midpoint of a line segment). 


1 


understand and use appropriate 
counting procedures to determine 
probabilities 


3 


IIF.l. know formal notation (e.g., sigma 
notation and factorial notation). 


2 


calculate and use the different 
measures of central tendency, 
variability, and range as appropriate 
to describe data 


2 


VIA.2. use variability and measures of central 
tendency (e.g., standard deviation, range, and 
mode) in external problems. 


3 


translate among tabular, symbolic, 
and graphical representations of 
relations using =, Ji, >, <, >, £ 


2 


IB. 1 . understand the uses of mathematical 
symbols as well as the limitations on their 
appropriate uses (e.g., equal signs, parentheses, 
superscripts, and subscripts). 


2 






IIC.3. represent functions, patterns, and 
relationships in different ways (e.g., statements, 
formulas, and graphs). 


3 


use variables to write expressions, 
equations, and inequalities 


3 


IA.7. use the correct order of arithmetic 
operations, particularly demonstrating facility 
with the Distributive Law. 


2 






IB. 1. understand the uses of mathematical 
symbols as well as the limitations on their 
appropriate uses (e.g., equal signs, parentheses, 
superscripts, and subscripts). 


2 






HE. 1 . recognize which type of expression best 
fits the context of a basic application (e.g., linear 
equation to solve distance/time problems; 
quadratic equation to explain the motion of a 
falling object; or compound interest as an 
exponential function). 


3 


simplify and evaluate expressions 
and formulas 


1 


IA.2. use exponents and scientific notation. 
IA.3. use radicals correctly. 


1 

2 


search systematically for patterns in 
complex situations 


3 


VG.3. show an understanding of how to modify 
patterns to obtain different results. 


4 


use multiple strategies 


3 


VG. 1 . are willing to experiment with problems 
that have multiple solution methods. 


'4 






VG.4. show an understanding of how to modify 
solution strategies to obtain different results. 


4 


apply viable strategies and 
appropriate concepts and procedures 
to construct a solution 


4 


VA.l. use inductive reasoning in basic 
arguments. 

VA.2. use deductive reasoning in basic 
arguments. 


3 

3 


test conjectures by formulating a 
proof or by constructing a 
counterexample 


4 


VA.2. use deductive reasoning in basic 
arguments. 


3 
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K-IO Mathematics EALRs 


POK 

EALR 

Rating 

( 1 - 5 ) 


University-level KSUS 


DOK 

KSUS 

Rating 

( 1 - 5 ) 






VA.7. understand the uses of both proof and 
counterexample in problem solutions and are 
able to conduct simple proofs. 


4 


support arguments and justify results 
using inductive and deductive 
reasoning 


4 


VA.l. use inductive reasoning in basic 
arguments. 


3 


express complex ideas and situations 
using mathematical language and 
notation in appropriate and efficient 
forms 


3 


VB.l. translate simple statements into 
equations (e.g., “Bill is twice as old as John” 
expressed by the equation b=2j). 

VB.2. understand the role of written symbols in 
representing mathematical ideas and the precise 
use of special symbols of mathematics. 


2 

2 


explain or represent complex 
mathematical ideas and information 
in ways appropriate for audience and 
purpose 


4 


IB.l. understand the uses of mathematical 
symbols as well as the limitations on their 
appropriate uses (e.g., equal signs, parentheses, 
superscripts, and subscripts). 


2 






lie. 3. represent functions, patterns, and 
relationships in different ways (e.g., statements, 
formulas, and graphs). 


3 






VB.2. understand the role of written symbols in 
representing mathematical ideas and the precise 
use of special symbols of mathematics. 


2 


extend mathematical patterns and 
ideas to other disciplines 


4 


VH.l. know that mathematical applications are 
used in other fields (e.g. carbon dating, 
exponential growth, predator/prey models, 
periodic motion and the interactions of waves, 
and amortization tables). 


2 


apply mathematical thinking and 
modeling in other disciplines 


4 


VH.l. know that mathematical applications are 
used in other fields (e.g. carbon dating, 
exponential growth, predator/prey models, 
periodic motion and the interactions of waves, 
and amortization tables). 


2 


identify situations in which 
mathematics can be used to solve 
problems with local, national, or 
international implications such as 
calculating resources necessary for 
interstate highway maintenance 


3 


VH.l. know that mathematical applications are 
used in other fields (e.g. carbon dating, 
exponential growth, predator/prey models, 
periodic motion and the interactions of waves, 
and amortization tables). 


2 
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Appendix C: Concurrence and Depth of Knowledge Between 10*'' Grade 



English/Language Arts EALRs and ] 


<^SUS English Standards 


lOth-Grade Reading EALRs 


DOK 

EALR 

Rating 

(1-5) 


University-level KSUS 


DOK 

KSUS 

Rating 

(1-5) 


readily use a variety of strategies to 
comprehend words and ideas in 
complex texts including self- 
correcting, re-reading, reading-on, and 
slowing down 


3 


IB.2. use monitoring and self-correction, as 
well as reading aloud, as means to ensure 
comprehension. 


2 


understand and apply reading 
strategies including word origins, 
word roots, prefixes, suffixes; making 
predictions; and verifying and revising 
understanding while reading 


3 


IA.6. exercise a variety of strategies to understand 
the origins and meanings of new words, including 
analysis of word roots and affixes; recognition of 
cognates and contextual clues; and the 
determination of word derivations. 


2 






IB.4. exercise a variety of strategies to understand 
the origins and meanings of new words, including 
the analysis of word roots and affixes; recognition 
of cognates and contextual clues; and the 
determination of word derivations. 


2 


identify literary devices (exaggeration, 
irony, humor, dialogue, devices that 
develop characterization, tension, and 
mood) 


2 


IA.2. recognize and comprehend narrative 
terminology and techniques, such as author versus 
narrator, historical versus implied author, and 
historical versus present-day reader. 


2 






IA.4. understand plot and character development in 
literature, including character motive, causes for 
actions, and the credibility of events. 


3 


analyze literary elements (plot, 
characters, setting, theme, point of 
view, conflict, resolution) 


3 


lA. 1 . engage in an analytic process to enhance 
comprehension and create personal meaning when 
reading text. This includes the ability to annotate, 
question, agree or disagree, summarize, critique, 
and formulate a personal response. 


3 






IA.4. understand plot and character development in 
literature, including character motive, causes for 
actions, and the credibility of events. 




read, analyze, and use informational 
materials to demonstrate 
understanding and expertise; analyze 
the validity of electronic information 


3 


IB.l. understand instructions for software, job 
descriptions, college applications, historical 
documents, government publications, newspapers, 
and textbooks. 


5 






IIIB.5. are able to evaluate sources of information 
located on the Worldwide Web in particular to 
ascertain their credibility, origin, potential bias, and 
overall quality. 


3 


synthesize ideas from selections to 
make predictions and inferences about 
various texts 


4 


IA.7. make supported inferences and draw 
conclusions based on textual features, seeking such 
evidence in text, format, language use, expository 
structures, and arguments used. 


3 
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draw conclusions based on the 
validity and accuracy of what is read 


4 


IA.7. make supported inferences and draw 
conclusions based on textual features, seeking such 
evidence in text, format, language use, expository 
structures, and arguments used. 


3 


organize information from resource 
materials and communicate findings 
effectively 


3 


IIIA.l. formulate research questions, refine topics, 
develop a plan for research, and organize what is 
known about the topic. 


4 


locate, analyze, and interpret material 
to investigate a question, topic, or 
issue (encyclopedia and other 
reference materials, pamphlets, book 
excerpts, newspaper and magazine 
articles, letters to an editor, etc.) 


3 


IIIA. l. formulate research questions, refine topics, 
develop a plan for research, and organize what is 
known about the topic. 

IIIB. l. collect information to develop a topic and 
support a thesis. 


4 

3 


read, analyze, and interpret a full 
range of texts fluently (instructions, 
news articles, poetry, novels, short 
stories, professional-level materials 
that match career or academic 
interests, electronic information, etc.) 


3 


IIIB.3. use a variety of primary and 
secondary sources, print or electronic, 
including books, magazines, newspapers, 
journals, periodicals, and the Internet. 


2 


understand and follow complex 
information to perform tasks for a 
specific audience (schedules, maps, 
recipes, instructions, newspaper want 
ads, consumer reports, travel books, 
first aid or other manuals, catalogs, 
yellow pages, credit card or job 
applications, legal documents, etc.) 


3 


IB.l . understand instructions for software, job 
descriptions, college applications, historical 
documents, government publications, newspapers, 
and textbooks. 


2 


read, respond to, and evaluate a 
variety of traditional and 
contemporary literature (poetry, 
essays, short stories, novels, 
biographies, non-fiction narratives, 
plays) 


3 


lA.l. engage in an analytic process to enhance 
comprehension and create personal meaning when 
reading text. This includes the ability to annotate, 
question, agree or disagree, summarize, critique, 
and formulate a personal response. 

IA.3. use reading skills and strategies to understand 
a variety of types of literature, such as epic pieces 
(for instance, Iliad) and lyric poems, as well as 
narrative novels and philosophical pieces. 


3 

2 


use appropriate reading strategies for 
interpreting technical and non- 
technical documents from different 
career settings such as scanning, 
finding specific information, and 
inferring from data 


3 


IB. 2. use monitoring and self-correction, as 
well as reading aloud, as means to ensure 
comprehension. 


2 
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Appendix D: Concurrence and Depth of Knowledge Between 10*'' Grade Science 
EALRs and KSUS Natural Science Standards 



K-10 Science EALRs 


DOK 

EALR 

Rating 

(1-5) 


University-level KSUS 


DOK 

KSUS 

Rating 

(1-5) 


describe the average speed, direction 
of motion, and average acceleration 
of objects, /or example increasing, 
decreasing, or constant acceleration 


2 


6B.1. understand Newton’s laws as a classical 
description of motion. For example that: 

• a force is required to alter an object’s motion 

• in the absence of force, or when forces are 
balanced, no change in motion is observed 

• forces are additive and the motion of an object 
is determined by the cumulative effect 


3 


understand many forms of energy as 
they are found in common situations 
on earth and in the universe 


2 


6A.1. understand the relationship between heat 
and temperature. For example: 

• that heat energy consists of the random motion 
and vibrations of atoms, molecules, and ions 

• the higher the temperature, the greater the 
atomic or molecular motion 


3 






6A.4. understand the distinction between kinetic 
(thermal, translational, and vibrational) and 
potential (gravitational and electrostatic) energy. 


2 


understand that total energy is 
conserved; analyze decreases and 
increases in energy during transfers, 
in terms of total energy conservation 


3 


6A.2. understand the conservation of energy and 
the First Law of Thermodynamics (i.e., energy 
cannot be created or destroyed but only changed 
from one form to another) and understand that 
energy must be transferred via work or heat. 


3 






6A.5. understand how energy can be transferred 
from one form to another. 


5 


explain how patterns and 
arrangements of landforms, oceans, 
and atmosphere are determined by 
natural forces and how the theory of 
plate tectonics accounts for 
movement over time 


3 


3A.1. know that the earth is a body in space 
whose environmental system (the atmosphere, 
lithosphere, cryosphere, hydrosphere, and 
biosphere) depends largely on the sun for light 
and heat, and that the current environment (e.g., 
geography and climate) is subject to change. 


3 






3B.1. are familiar with the history of the Earth. 


2 


describe how genetic information 
(DNA) in the cell is controlled at the 
molecular level, and provides genetic 
continuity between generations 


2 


4B.1. know the chemical and structural 
properties of DNA in heredity and protein 
synthesis (e.g., DNA synthesis, transcription, 
translation; mRNA and the genetic code; and 
effect of mutations). 


2 






4B.2. understand Mendel’s laws of heredity 
(e.g., genes and alleles; genotype versus 
phenotype; segregation and independent 
assortment; and dominant versus recessive 
traits). Understand how Mendel's laws relate to 
the movement of chromosomes to gametes 
during meiosis, and understand the 
chromosomal basis of sex determination. 


3 
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K-10 Science EALRs 


DOK 

EALR 

Rating 

(1-5) 


University-level KSUS 


DOK 

KSUS 

Rating 

(1-5) 


understand that patterns of movement 
in the plates that comprise the earth’s 
surface are the result of outward 
transfer of the earth’s internal heat, 
and that historical patterns of 
movement can be identified from 
clues in rock formations; describe 
how volcanoes and earthquakes in 
Washington State occur because of 
this interaction 


2 


3B.3. understand the processes of volcanism and 
erosion. 


3 


understand that the earth, planets, 
sun, and the rest of the celestial 
bodies in the universe are continuing 
to evolve because of interactions 
between matter and forces of nature 


2 


3A.1 . know that the earth is a body in space 
whose environmental system (the atmosphere, 
lithosphere, cryosphere, hydrosphere, and 
biosphere) depends largely on the sun for light 
and heat, and that the current environment (e.g., 
geography and climate) is subject to change. 


3 






3B.1. are familiar with the history of the Earth. 


2 






3B.2. are familiar with the history of the solar 
system. 


2 


explain how organisms can sustain 
life by obtaining, transporting, 
transforming, releasing, and 
eliminating matter and energy 


3 


4A.12. know that in order to be alive, cells must 
exchange materials with their environment 
and/or with other cells. 

4A.13. know that such exchanges involve a 
variety of mechanisms for transporting materials 
across a membrane, including diffusion, 
osmosis, and transport involving specialized 
membrane proteins 


2 

3 






4.C.I. know that multicellular organisms have a 
variety of specialized cells, tissues, organs, and 
organ systems that each perform specialized 
functions (e.g., digestion, respiration, 
circulation, excretion, movement, control and 
coordination, protection from disease, and 
reproduction). Understand that the different 
organ systems are integrated to make 


3 


investigate and examine the scientific 
evidence used to develop theories for 
evolution, speciation, adaptation, and 
biological diversity 


4 


4D.2. understand the concept of natural selection 
(differential survival and reproduction of chance 
inherited variants, depending upon 
environmental conditions). 


2 






4D.3. understand the theory of evolution (e.g., 
the Earth’s present-day life forms evolved from 
earlier, distinctly different species). Know that 
genetic change among individuals of populations 
is the raw material for evolution 


3 
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K-10 Science EALRs 


DOK 

EALR 

Rating 

(1-5) 


University-level KSUS 


DOK 

KSUS 

Rating 

(1-5) 


compare and contrast the complex 
factors (biotic and abiotic) that affect 
living organisms’ interactions in 
biomes, ecosystems, communities, 
and populations 


3 


3C.1. understand the notion of habitats and their 
role in evolution. 

4D.2. understand the concept of natural selection 
(differential survival and reproduction of chance 
inherited variants, depending upon 
environmental conditions). 


3 

2 


study and analyze questions and 
related concepts that guide scientific 
investigations 


3 


lA.l. design and conduct scientific 
investigations during which they formulate and 
test hypotheses (formulate and clarify the 
method; identify the controls and variables; 
collect, organize, display, and analyze data; 
make revisions of hypotheses, methods, and 
explanations; present the results; and seek 
critiques from others). 


4 


use mathematics, computers and/or 
related technology to model the 
behavior of objects, events, or 
processes 


3 


lA.l. design and conduct scientific 
investigations during which they formulate and 
test hypotheses (formulate and clarify the 
method; identify the controls and variables; 
collect, organize, display, and analyze data; 
make revisions of hypotheses, methods, and 
explanations; present the results; and seek 
critiques from others). 


4 






1F.2. understand that a curve drawn in a certain 
location is fully equivalent to a set of algebraic 


3 


research, interpret, and defend 
scientific investigations, conclusions, 
or arguments; use data, logic, and 
analytical thinking as investigative 
tools; express ideas through oral, 
written, and mathematical expression 


4 


lA.l. design and conduct scientific 
investigations during which they formulate and 
test hypotheses (formulate and clarify the 
method; identify the controls and variables; 
collect, organize, display, and analyze data; 
make revisions of hypotheses, methods, and 
explanations; present the results; and seek 
critiques from others). 


4 






2.1. understand and use data represented in 
various ways (e.g., charts, tables, plots, and 
graphs). 


3 
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K-IO Science EALRs 


DOK 

EALR 

Rating 

(1-5) 


University-level KSUS 


DOK 

KSUS 

Rating 

(1-5) 


analyze and explain why curiosity, 
honesty, openness, and skepticism 
are integral to scientific inquiry 


3 


2A.1. understand that science and the theories of 
science are not absolute and should be 
questioned and challenged. This includes the 
ideas that: 

• new theories will continue to replace current or 
older ones 

• scientific theories must stand up to the scrutiny 
of the entire scientific community 

• acceptable validation includes reproduction 
and internal consistency 


4 






2A.5. know that investigations and public 
communication among scientists must meet 
certain criteria in order to result in new 
understanding and methods. For example: 

• arguments must be logical and demonstrate 
consistency between natural phenomena 
revealed by investigations, as well as the 
historical body of scientific evidence 

• the methods and procedures used to obtain 
evidence must be clearly reported and 
reproducible to enhance opportunities for further 
investigation 


3 


identify and analyze factors that limit 
the extent of scientific investigation 


3 


2A.1. understand that science and the theories of 
science are not absolute and should be 
questioned and challenged. This includes the 
ideas that: 

• new theories will continue to replace current or 
older ones 

• scientific theories must stand up to the scrutiny 
of the entire scientific community 

• acceptable validation includes reproduction 
and internal consistency 


4 


compare, contrast, and critique 
divergent results from scientific 
investigations based on scientific 
arguments and explanations 


3 


2A.1. understand that science and the theories of 
science are not absolute and should be 
questioned and challenged. This includes the 
ideas that: 

• new theories will continue to replace current or 
older ones 

• scientific theories must stand up to the scrutiny 
of the entire scientific community 

• acceptable validation includes reproduction 
and internal consistency 


4 

/ 

'1 






2A.5. know that investigations and public 
communication among scientists must meet 
certain criteria in order to result in new 
understanding and methods. For example: 

• arguments must be logical and demonstrate 
consistency between natural phenomena 
revealed by investigations, as well as the 
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DOK 




DOK 


K-10 Science EALRs 


' iiiAL/K 
Rating 


University-level KSUS 


KSUS 

Rating 




(1-5) 




(1-5) 






historical body of scientific evidence 
• the methods and procedures used to obtain 
evidence must be clearly reported and 
reproducible to enhance opportunities for further 
investigation 


4 






lA.l. design and conduct scientific 
investigations during which they formulate and 
test hypotheses (formulate and clarify the 
method; identify the controls and variables; 
collect, organize, display, and analyze data; 
make revisions of hypotheses, methods, and 
explanations; present the results; and seek 
critiques from others). 




analyze and evaluate the quality and 


3 


lA.l. design and conduct scientific 


4 


standards of investigative design, 




investigations during which they formulate and 




processes, and procedures 




test hypotheses (formulate and clarify the 
method; identify the controls and variables; 
collect, organize, display, and analyze data; 
make revisions of hypotheses, methods, and 
explanations; present the results; and seek 
critiques from others). 


3 






2A.4. know that scientists throughout history 
have had many difficulties convincing their 
contemporaries to acknowledge what are now 
generally accepted scientific ideas 




know that science involves testing. 


3 


lA.l. design and conduct scientific 


4 


revising, and occasionally discarding 




investigations during which they formulate and 




theories; understand that scientific 




test hypotheses (formulate and clarify the 




inquiry and investigation lead to a 




method; identify the controls and variables; 




better understanding of the natural 




collect, organize, display, and analyze data; 




world and not to absolute truth 




make revisions of hypotheses, methods, and 
explanations; present the results; and seek 
critiques from others). 


3 






2A.2. know ways in which science and society 
influence each other. 

For example that: 

• scientific methods and the knowledge they 
produce may influence how people think about 
themselves and their world 

• technology can contribute to the solution of an 
individual or community problem 

• social and economic forces strongly influence 
which science and technology programs are 
pursued, invested in, and used 
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